Big Data Technology Literature Review

Storing and manipulating Big Data relies on various data structures, algorithms and technologies. Some of these are new, while others have existed for quite a while (the Bloom filter was presented in 1970) and are now making their way into mainstream software engineering. We present those algorithms and technologies briefly, and provide citations for further reading. This paper is organized as follows: First we discuss relevant theoretical background, including algorithms and design approaches. Then we look at concrete technologies and a few example products. Many Big Data products exist today, and new ones are being added constantly. Thus, products mentions below should be treated as concrete examples, rather than recommendations. Each topic ends with a “further reading” section, for those who wish to deepen their knowledge.